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Abstract. The LIGO Scientific Collaboration (LSC) is developing and running 
analysis pipelines to search for gravitational-wave transients emitted by astrophysical 
events such as compact binary mergers or core-collapse supernovae. However, because 
of the non-Gaussian, non-stationary nature of the noise exhibited by the LIGO 
detectors, residual false alarms might be found at the end of the pipelines. A critical 
aspect of the search is then to assess our confidence for gravitational waves and to 
distinguish them from those false alarms. Both the "Compact Binary Coalescence" 
and the "Burst" working groups have been developing a detection checklist for the 
validation of candidate-events, consisting of a series of tests which aim to corroborate 
a detection or to eliminate a false alarm. These tests include for example data quality 
checks, analysis of the candidate appearance, parameter consistency studies, and 
coherent analysis. In this paper, the general methodology used for candidate validation 
is presented. The method is illustrated with an example of simulated gravitational- 
wave signal and a false alarm. 
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1. Introduction 

The Compact Binary Coalescence (CBC) working group of the LIGO-Virgo joint 
collaboration is a data-analysis group looking for gravitational-wave signals emitted by 
inspiralling compact binary systems [H El El Hj . The time that such a signal will spend in 
the frequency bandwidth of the current ground-based gravitational- wave interferometers 
lies in a range which goes from a few tenths up to several tens of seconds. 

The Burst working group of the LIGO-Virgo joint collaboration is a data-analysis 
group looking for transient gravitational-wave signals without specific assumptions on 
the expected waveform [SIE]. The typical duration of the signals searched by the Burst 
group is of the order of the millisecond to a tenth of second. 

Both of these working groups use analysis methods that are sensitive to the non- 
stationary and non-gaussian noise of the detectors. Noise transients can induce false 
alarm triggers in the analysis pipelines that may result in accidental coincidences 
between the interferometers. It is therefore crucial to submit each gravitational- wave 
candidate identified by the analysis to a detection checklist which aims to estimate 
confidence in this candidate. 

The detection checklist is made of a list of standard tests in different stages of 
development that are used to review the gravitational- wave candidates. An overview 
of this checklist is provided in Section [21 Section [3] shows a method implemented by 
the Burst and CBC groups to estimate the statistical significance of the candidates. 
In Section IH the paper will describe with more details a few items of the detection 
checklist, using an example of simulated gravitational- wave signal and false- alarm trigger 
for illustration purposes. 

2. Overview of the detection checklist 

A detection checklist to evaluate the significance of candidate-events has been developed 
by both the CBC and Burst groups. Despite some specificities inherent to the kind 
of signals that are being looked for by the CBC and Burst searches, the method 
implemented to estimate confidence in a gravitational-wave candidate is very similar 
for both data-analysis groups. Thus the summary of the checklist provided in this 
section applies both to the CBC and Burst groups, unless otherwise noted. 

The list presented below is a short synthesis of the tests implemented in the CBC 
and Burst detection checklists. As many of these tests are still under development 
or refinement, the checklist is rapidly evolving, and the following list should not be 
considered exhaustive. Moreover, it is expected that many of the tests that are at 
present qualitative and based on the experience gained about the instruments will be 
developed into quantitative tests. Here we outline the main tests that are currently part 
of the detection checklist for candidate-events or in the process of implementation: 

• Statistical significance The first step of the candidate validation procedure 
consists of determining the statistical significance of the candidates identified by 
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the analysis pipeline, that is to say the probability of coincident triggers arising 
from random coincidences of noise triggers. The general method is described in 
Section [3l which will also explain how the statistical significance of the candidates 
affects the way the other tests of the checklist are addressed. 

• Data integrity: sanity checks to verify that the data set containing the candidate 
is not corrupted. 

• Status of the interferometers: The state of the LIGO interferometers [1] and 
their sensitivity near the time of the candidate are checked. This test also includes 
a verification of the data quality flags recorded in the database. Section 14.11 will 
show how this test can allow the identification of noisy data segment containing 
false-alarm triggers. 

• Environmental or instrumental causes: We analyze auxiliary channels of the 
interferometers, such as the environmental sensors or the signals involved in the 
mirror control loops [Ij , to check for the presence of possible noise transients which 
could be the cause of a false alarm identified by the analysis. This effort is also 
part of the "glitch group" activities [7] . More details on this part of the detection 
checklist are provided in Section 14. 2[ 

• Candidate's appearance: Part of the detection checklist consists of tests of the 
candidate's appearance. These tests aim to confirm the presence of a gravitational- 
wave signal in the data or to identify obvious excesses of noise responsible for 
a false alarm. A variety of tools are used to examine the data containing the 
candidate-event, such as time series, time-frequency spectrograms, or the outputs 
of the search pipeline, namely the signal to noise ratio (SNR) or the [H] time 
series in the case of the CBC search [9]. The analysis of this graphical information is 
performed by comparing the results obtained at the time of the candidate-event to 
the expectations for simulated gravitational-wave signals or for known instances of 
false-alarm triggers. Two examples of candidate appearance tests will be presented 
in Section 14.31 A complementary and quantitative test to establish a likelihood 
ranking of the candidates given their estimated parameters is in preparation. 

• Detection robustness: The Burst group has developed many independent search 
algorithms looking for unmodelled gravitational-wave signals, a few instances of 
which are Block-Normal [10], KleineWelle [IT], QOnline [12] and Waveburst [T3] . 

The robustness of a detection for burst searches is checked by verifying that the 
candidate-event is identified by different independent algorithms. In the case of the 
CBC search, the list of candidate-events is obtained from a single analysis pipeline 
based on match filter algorithm [9] using specific inspiral waveforms [Hj. However 
a Bayesian analysis dedicated to the search for inspiral signals [15] has also been 
developed and the CBC group has started using it as an independent algorithm to 
check the robustness of the detection. Part of the detection robustness test also 
consists in verifying the accuracy of the detectors' calibration [T6l [T7] at the time 
of the detection. A current effort of the CBC group is to automate a test that will 



Detection Confidence Tests for Burst and Inspiral Candidate Events 



4 



check the impact of possible cahbration errors on the analysis. To this purpose 
the data will be reanalyzed using a calibration modified to represent the possible 
uncertainties on the measurement. 

• Correlation between interferometers: Other tests of the detection checklist 
consist in checking for the correlation between the signals measured in the 
different detectors of the network composed of the three LIGO interferometers [1], 
GEO600 [I8l and Virgo [2]. The Burst group has developed many network or 
coherent analyses [19], one example of which is Coherent Waveburst, based on a 
constraint likelihood method [SniEI]- The CBC group is developing two tests that 
check for the expected signal correlations in the data containing coincident triggers 
from multiple detectors. One of these tests uses the matched filtering algorithm [9] 
to compute the multi-interferometer coherent SNR [22l [23] and compares it with 
the null-stream statistic [23] for inspiral waveforms to assess the significance of a 
trigger being an inspiral event. The essential idea behind this test is that above a 
certain threshold value for the coherent SNR, real gravitational-wave signals will 
yield a smaller value for the null-stream statistic than instrumental or environmental 
glitches of the same coherent SNR. The second test uses a Bayesian approach to 
infer the posterior distributions of the signal parameters [25]. Both are currently 
tested as part of the detection checklist. In addition to these checks for correlation 
between interferometers, the analysis groups also verify whether the candidate-event 
is identified by the resonant bar detectors [26] . 

• Coincidence with other types of astrophysical observatories: The 

LIGO Scientific Collaboration is implementing procedures to check for possible 
coincidences between gravitational-wave candidates and Gamma-Ray Bursts, 
optical transients or neutrinos observations [19] . 



3. Statistical significance of the candidates 

In order to estimate the statistical significance of the candidate-events, the CBC and 
Burst searches compare the triggers found in coincidence between at least two of the 
LIGO interferometers to an expected background of accidental coincidences. This 
background is estimated by repeating the analysis after time-shifting the data of each 
interferometer with respect to each other. This method, called the time-slides analysis, 
has already been described in previous publications such as [27]. Figure shows 
an example of comparison between the in-time coincident candidate-events and the 
expected background for the Binary Neutron Star search run over the data taken during 
the LIGO fourth (S4) science run p7]. In this example, the background was estimated 
with one hundred time-slide experiments. The goal of a candidate's follow-up with 
the checklist depends on whether or not the candidate has a low probability of being 

I Reprinted figure with permission from Abbott B. et al (LIGO Scientific Collaboration), 
Phys. Rev. D Vol. 77, 062002 (2008). Copyright 2008 by the American Physical 



Society. URL: http://link.aps.org/abstract/PRD/v77/e062002 
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Figure 1. Cumulative histogram of the combined SNR, pc, for the S4 Binary 
Neutron Star search: for the in-time coincident candidate-events (triangles), and for 
the estimated background of accidental coincidences (crosses and 1 standard-deviation 
ranges) . This figure has been extracted from [57] . All candidates were found consistent 
with the background. 



an accidental coincidence. If the loudest candidate is consistent with the estimated 
background (as the candidates shown in figured]), then the few loudest candidates are 
investigated to make sure that the accidental coincidences are not due to an undiagnosed 
instrumental artefact. When such an instrumental artefact is diagnosed properly, this 
can lower the background and reveal gravitational waves. In case a candidate has a low 
probability of accidental coincidence (i.e. if the candidate is lying above the estimated 
background), the goal of the follow-up is then to strengthen our confidence in a possible 
detection by submitting the candidate to the detection checklist that gravitational-wave 
signals should pass successfully. 

4. Detailed examples of the checklist 

This section will highlight a few examples of tests used for the review of candidate- 
events. In order to illustrate the expected results for an inspiral gravitational- wave 
signal we will refer to a simulated inspiral signal. This simulation was performed by 
acting on one of the arm test masses of the interferometers to generate a differential 
motion of the interferometer arm cavities approximately equivalent to the expected 
effect of a gravitational wave. This simulated gravitational- wave signal was detected by 
the CBC analysis and stands as an outlier above the estimated background. We will 
also illustrate the behaviour of the detection checklist when it is applied to an example 
of false alarm. In the following subsections, we will refer to the simulated gravitational 
wave as Candidate G and to the false alarm as Candidate F. 
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Figure 2. Inspiral range as a function of time at Livingston (the origin of time is 
arbitrary) . The dashed vertical lines indicate the start and end times of the segments 
when the interferometer was in science mode. Two science segments are represented in 
this figure (corresponding to the time windows 16.5-102.2 min and 122.7-162.9 min). 
The Candidate F was found in the first science segment, at t~98.2 min. This time is 
underlined by a solid vertical line. 



4.1. Status of the interferometers 

A fraction of the detection checklist consists in verifying the status of the interferometers 
and the data quahty in the segment containing the candidate. This includes examining 
the figures of merit (detectors state, sensitivity, seismic trends) posted in the detectors 
log, scanning the database which contains the hst of data quality flags, as well as checking 
the information reported by the "glitch group" [7] . The goal of this study is to check 
for a possible misbehaviour of the detectors or an unusual excess of noise which could 
translate into a higher rate of false-alarm triggers and thus reduce our confidence in the 
candidate-event. For instance we check how the detectors sensitivity varies in time and 
how it might affect the performances of our searches. 

Figure [2] shows a plot displaying the minute trends of the inspiral search range at 
the Livingston's site, called inspiral range |28], for a time window of about 3 hours which 
includes the Candidate F. The inspiral range is conventionally defined as the distance 
at which the coalescence of a 1.4-1.4 Mq binary neutron stars system would be detected 
by the search with a SNR of 8 averaged over all sky positions and orientations. During 
the day from which the three hours of data shown in figure [2] have been extracted, 
the typical inspiral range measured in science mode was fluctuating between 8.5 and 
10.5 Mpc. However one can notice that the first science segment shown in figure [2] 
(between t=16.5 min and t=102.2 min) ends with a sharp drop in the inspiral range 
for about twenty five minutes. The inspiral trigger associated with Candidate F (false 
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alarm) in the Livingston data was found within this segment (it is highhghted by the 
sohd vertical line in figure [2]) while the inspiral range was about 2.5 Mpc, that is to say 
well below the averaged sensitivity reached by the interferometer during this day. At 
these times, the background is usually significantly larger than the average during the 
run used to calculate the statistical significance of the candidate. Ranking the science 
segments by the standard deviation of the inspiral range, this segment was found to be 
in the top 0.5%, indicating a very noisy time. Since the Candidate Fwas first examined, 
a data quality fiag has been created to mark such noisy segments (this is an advisory 
fiag used to exert caution in case of a detection candidate). The presence of an unusual 
excess of noise in the Livingston interferometer was also reported in the detector log by 
the control room experts which were monitoring the detector's behaviour at that time. 
The remarkable noisiness of the detector prejudices us against Candidate F, although 
this check does not prove that the candidate itself is due to detector noise. An evidence 
of the nature of the candidate is brought by its SNR or [H] time series as discussed 
in Section 1131 

4-2. Environmental and instrumental causes 
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Figure 3. Q spectrogram of a seismometer channel located near the end mirror 
test mass of the Hanford 2 km interferometer. The loudest transient found in this 
spectrogram is located at low frequency (f~2.6 Hz) and is time-coincident with the 
inspiral trigger associated to the Candidate G (simulated gravitational-wave signal) 
whose position corresponds to the origin of the x axis (t=0 s). 

In order to check for possible instrumental artefacts that could be responsible for 
false alarm triggers, we examine the auxiliary channels of the detectors in a few seconds 
long window around the candidate-events. For this purpose time-frequency maps of 
auxiliary channels are being analyzed, using an event visualization tool called QScan, 
which is based on a Q-transform [12] . Qscan produce time-series and Q spectrograms of 
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Figure 4. Q spectrogram of the error signal of tlie differential mode control loop of 
the Hanford 2 km interferometer at the time of the detection of Candidate G (this is 
the main port sensitive to gravitational waves). The origin of the x axis (t=0 s) is the 
same as in figure [3] 



the auxiliary channels in which transients are detected. The Q spectrograms correspond 
to time-frequency decompositions using sinusoidal Gaussians characterized by a central 
time, central frequency, and a quality factor Q. An example of Q spectrogram is provided 
in figure [3|, where the examined channel is a seismometer located near the end mirror 
test mass of the Hanford 2 km interferometer. This kind of channel can measure seismic 
disturbances at frequencies below a few Hertz. In figure [3] a low frequency transient 
(approximately at 2.6 Hz) lasting for a few seconds (between t~-2 s and t~4 s along 
the time axis) is visible. Moreover this seismometer's transient is coincident with the 
Candidate C (simulated gravitational-wave signal) whose time corresponds to t=0 s in 
figure [HI When a candidate-event is simultaneous with a transient in an auxiliary chan- 
nel, further investigations are performed as explained below. 

Statistical significance of instrumental transients 

A transient found in an auxiliary channel is characterized by a parameter called Z signif- 
icance [12], equivalent to a SNR squared for the Q-transform. Under the assumption of 
Gaussian noise, the Z significance could be associated to a false alarm rate proportional 
to e~^ . However, due to the non-Gaussian, non- stationary nature of the noise present 
in the auxiliary channels, we cannot rely on the previous assumption. Therefore, in 
order to obtain a statistical ranking of an instrumental transient, we need to determine 
empirically the background of the auxiliary channel containing this transient. 

For each auxiliary channel the background is estimated by running QScan at 
times which are randomly distributed over data-taking epochs and by recording the 
Z significance measured at each of these random times. This provides a distribution 
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Figure 5. Scatter plot of QScan significance in a seismometer channel (x axis) and in 
the error signal of the differential mode control loop (y axis) of the H2 interferometer. 
The "circles" refer to the QScan significance measured at times randomly distributed 
over the first calendar year of the fifth LIGO science run ("S5") to estimate 
the seismometer background. The "cross" symbol refers to the QScan significance 
measured at the time of the Candidate G (simulated gravitational- wave signal). 



of Z significances corresponding to the background of the auxihary channel. Since the 
background of an auxihary channel might change during a long data-taking due for 
example to seasonal variations of the environmental noises or to instrumental drifts, we 
perform several estimations of the background, over large (such as a year) and short 
(such as a few days) epochs, and compare the results. 

Once the auxiliary channel background has been estimated, we can then compare 
the Z significance measured at the time of the analyzed candidate to this background, 
and thus obtain a statistical ranking of the instrumental transient. Figure [5] shows a 
scatter plot of the Z significance measured in the seismometer {x axis) versus the Z 
significance measured in the error channel of the differential mode control loop of the 
H2 interferometer [y axis), which is the main port sensitive to gravitational waves. 
The "circles" refer to the background estimated from about 1000 times randomly 
distributed over the first calendar year of the fifth LIGO science run ("S5") [1] while 
the "cross" symbol shows the significances measured in both channels at the time 
when the Candidate C was detected. A seismic transient whose Z significance would 
be comparable with the median of the seismometer background distribution could be 
ignored as irrelevant. On the contrary the seismic transient shown in figure [3] has a Z 
significance (read on the x axis) higher than 99.7% of the background, which makes it 
statistically relevant. Another estimation of the seismometer background was obtained 
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by analyzing only two days of data including the time of the Candidate G and led to a 
similar conclusion. 

The fact that a statistically loud seismic transient is found at the time of the Candi- 
date G is however not sufficient to rule it out as a possible detection. The next question 
that needs to be addressed is to identify whether or not this environmental transient 
couples to the interferometer output port. 

Coupling of environmental disturbances into the interferometer output port 

The coupling of an environmental disturbance into the interferometer output port can 
be proven by comparing the Q spectrogram of the auxiliary channel to the Q spectrogram 
of the interferometer output port and by looking for possible correlations between these 
two channels. 

A high frequency disturbance might couple linearly into the gravitational- wave 
bandwidth of the output port. Transfer functions have been measured between 
environmental channels (such as magnetometers, microphones, radio channels) and 
the interferometer output port [29]. These transfer functions can be compared to 
the amplitude ratio measured in the Q spectrograms of the auxiliary channel and the 
interferometer output port at the time of the candidate. If the amplitude ratio is 
consistent with the transfer function, this proves that the environmental disturbance 
coupled to the output port, and consequently leads to the rejection of the candidate as 
a possible detection. 

In the case of a low frequency seismic transient, noise up-conversion mechanisms 
might induce a false-alarm trigger in the gravitational-wave bandwidth. One can notice 
in figure [5] that the Z significance measured in the error signal of the interferometer 
differential mode (read on the y axis) at the time of the Candidate C is comparable to 
the significances obtained at random times. Therefore, despite the statistical relevance 
of the transient in the seismometer, figure [5] does not argue in favor of a possible 
coupling into the interferometer output port. Moreover the comparison between the 
Q spectrograms of the seismometer (figure [3]) and the error signal of the interferometer 
differential mode (figure Hj) does not indicate any correlation between these channels 
neither at the frequency of the seismic transient (2.6 Hz), neither at higher frequencies. 
Therefore it is unlikely that the seismic transient be the cause of the inspiral trigger 
associated to the Candidate g|§|. 

4-3. Candidate appearance 

In this section two examples of qualitative checks of the candidate's appearance are 
illustrated: a check of the candidate's time-frequency map, and a check of the output 
of the match-filtering algorithm [9] used to search for inspiral gravitational- wave signals. 

§ Knowing that the Candidate G is a simulated gravitational-wave signal, the presence of an inspiral 
trigger is indeed not related to the seismic transient. 
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Figure 6. Q spectrograms of the data containing tlie Candidate G (simulated 
gravitational-wave signal) in the two more sensitive detectors: Hanford 4 km (left) 
and Livingston 4 km (right). A "chirp" waveform (frequency increasing with time) is 
visible in the data of the two detectors. 



Q spectrograms of the candidate 

A QScan of the data in which a candidate-event as been detected is examined in or- 
der to perform the following checks: 

• The presence of a possible known signal waveform that might confirm the detection 
is verified. However, a low SNR inspiral signal is not expected to be visible in a Q 
spectrogram. Therefore the absence of visible known waveform in the spectrogram 
does not rule out a possible detection. 

• The presence of an obvious excess of noise in the data is also checked. 

Figure [6] shows the Q spectrograms of the Candidate G in the two more sensitive in- 
terferometers where this simulated gravitational-wave signal was injected. The tran- 
sient visible in the HI and LI data corresponds to the typical "chirp" pattern (fre- 
quency increasing with time) that is characteristic of an inspiral signal. The simulated 
gravitational-wave signal is thus visible in these two spectrograms. It is actually not 
clearly visible in the Q spectrogram of the H2 data (not represented here) because of the 
lower SNR in this interferometer. 

Output of the match-filtering algorithm 

Another example of check for the candidate's appearance that is used by the CBC group 
consists in examining the time-series of the SNR obtained after match-filtering [9j the 
data with inspiral waveforms [H], as well as the time-series of a which tests the 
consistency between the triggered waveform and the signal present in the data. 

An example of the expected time-series for a simulated gravitational-wave signal 
is shown in figure On the left plot, the SNR time series shows a short central peak 
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Figure 7. SNR (left) and (right) f8^, ^ time series obtained after match-filterinff 
the Livingston data containing the Candidate G (simulated inspiral gravitational- 
wave signal). The time origin on the x axis coincides with the time of the inspiral 
trigger. In the left plot the dashed horizontal line at SNR=5.5 corresponds to the SNR 
threshold used for this analysis. SNR peaks exceeding this threshold are recorded as 
inspiral triggers by the analysis pipeline. In the right plot, the time series shows a 
characteristic pattern for a few tens of milliseconds around the time of the Candidate 
G (t=0 s) which corresponds to the expectations for a gravitational-wave signal. In 
particular the is minimum at t=0 s when the triggered waveform best matches the 
signal present in the data. 




Time [s] Time [s] 



Figure 8. SNR (left) and (right) [U [9] time series obtained after match-filtering 
the Livingston data containing the Candidate F (false alarm). The time origin on 
the X axis coincides with the time of the inspiral trigger. In the left plot the dashed 
horizontal line at SNR=:5.5 corresponds to the SNR threshold, while, in the right plot, 
the dashed horizontal line at — 10 corresponds to the threshold used for this 
analysis. The inspiral triggers must exceed the SNR threshold while their x^ must be 
lower than the corresponding threshold. 

exceeding the threshold at SNR=5.5 used by this search. This SNR peak corresponds to 
the time of the trigger associated with the simulated inspiral signal. On the right plot, 
the time-series presents a very characteristic shape around the time of the inspiral 
trigger (t=0 s), which corresponds to the expectations for gravitational- wave signal in 
stationary gaussian noise. A few milliseconds before the time of the inspiral trigger 
the value starts increasing, while it falls to a minimum at t=0 s when the triggered 
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waveform best matches the simulated gravitational-wave signal injected in the data. 
Finally the time series presents a symmetrical behaviour after t=0 s. 

Notice that a veto based on the value of the at the time of the inspiral trigger 
(which would correspond to t=0 s in figure [7]) is already automatically implemented 
in the CBC analysis pipeline. This veto rejects any trigger whose value exceeds a 
threshold set to = 10 in the search that identified Candidate G and Candidate F. The 
tuning of this threshold tends to be very conservative to assure that real gravitational- 
wave events with waveforms that might differ slightly from the CBC searches templates 
are not rejected [30]. The qualitative check performed in the detection checklist is 
complementary to the veto as it consists in examining the time series for several 
tenths of seconds around the inspiral trigger. 

Figure M shows the SNR and time-series around the time of the Candidate F at 
Livingston. Multiple peaks of SNR exceeding the threshold are visible in the left plot, 
which indicates highly non-stationary data. In the right plot the time series does 
present a minimum at t=0 s which is the reason why this candidate was not vetoed by 
the analysis pipeline. However the x^ time series also shows large values for the whole 
two seconds window surrounding the candidate, which clearly differs from the plot shown 
in figure [7] and indicates a very noisy stretch of data. Accordingly the Candidate F can 
be ruled out as a possible detection, which confirms the first suspicions born from the 
analysis of the inspiral range in Section 14.11 

5. Conclusions and perspectives 

The Burst and CBC groups are pursuing the refinement of their respective detection 
checklists for candidate-event validation. Part of these tests are still under development 
and are expected to become more quantitative as experience about the instruments is 
gained. The groups are currently aiming to automate the detection checklist in order to 
build a candidate follow-up pipeline which will improve the swiftness of the analysis. The 
detection checklist is presently being applied to the candidates obtained by the searches 
analyzing the data taken during the fifth LIGO science run ("S5") jlj. The detection 
checklist should play an even more crucial role in the analysis of the future LIGO 
science runs, for which we expect better detectors' sensitivities and higher probabilities 
of gravitational-wave detections. 
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